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Abstract 

This article describes the design and implementation of external support to low- 
performing schools using data from Chicago and California. Using the literature on 
external support, instructional capacity, and policy strength, the study gathered data 
from interviews, observations, document review, and surveys. The findings suggest 
that the model of assistance employed in both Chicago and California was 
inadequate to the task. While the policies examined demonstrate recognition that 
low-performing schools need additional capacity if they are to substantially 
improve student outcomes, external support providers used limited and haphazard 
approaches, and as a result, the support component had little influence on teaching 
and learning. In addition, because the external supports relied on a market-like 
support structure with few other mechanisms to ensure quality, and because there 
was limited quantity (intensity) of support, the benefit that external assistance 
might otherwise have provided was limited. This was particularly problematic for 
the lowest capacity schools, many of which experienced limited change despite 
increased educator effort and involvement of external providers. In essence, 
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external assistance through these school accountability policies did little to improve 
educator and organizational performance. 

Keywords: accountability; educational policy; educational improvement 

La mejora de las escuelas de bajo desempeno a traves de ayudas externas: 
Lecciones de Chicago y California 

Resumen 

En este articulo se describe el diseno y aplicacion de ayudas externas a escuelas de 
bajo desempeno en Chicago y California. Este estudio recogio datos de entrevistas, 
observaciones, revision de documentos, encuestas y literatura sobre la ayudas 
externas , capacidad de instruction y fuerza politica. Los resultados sugieren que el 
modelo de asistencia que usaron Chicago y California fue inadecuado. Si bien las 
politicas examinadas reconocieron que las escuelas de bajo rendimiento necesitan 
mas apoyo si quieren mejorar sustancialmente los resultados de los estudiantes, la 
ayuda externa fue muy limitada, usaba los metodos de manera azarosa, y 
consecuentemente tuvo poca influencia en la mejora de la ensenanza y el 
aprendizaje. Ademas, las ayudas externas emplearon estructuras de apoyo de cuasi- 
mercados con pocos mecanismos para garantizar la calidad, y dado que la cantidad 
(intensidad) de asistencia era limitada, los beneficios que las ayudas externas serian 
muy limitados. Esto fue particularmente problematico para las escuelas con menor 
capacidad, a pesar del aumento de los esfuerzos de los educador y la participation 
de proveedores externos. Basicamente, la ayuda exterior a traves de estas politicas 
de rendition de cuentas (accountability) hicieron muy poco para mejorar la 
organization y el rendimiento escolar. 

Palabras clave: rendition de cuentas; politica educativa; mejora de la education. 


Through the No Child Left Behind Act of 2001 (NCLB), the federal government requires 
states to hold schools accountable for student performance. NCLB is based on an assumption that 
consequences will motivate school staff to perform at higher levels and focus attention on student 
outcomes. Once schools are identified as not making adequate yearly progress , districts must provide 
support and assistance to schools specifically in the areas of data analysis, identification and 
implementation of instmctional strategies, and budget analysis, and states must “create and sustain a 
statewide system of support” (U. S. Department of Education, 2006, p. 25). The law requires that 
states allocate 4% of their Title 1 funds for school improvement (see Public Law 107-110, Title 1, 
Section 1003). This focus on district and state assistance suggests an additional underlying 
assumption: Low student performance is not solely related to will but also to school and educator 
capacity. In mandating external support to low-performing schools, NCLB offers a public statement 
about the need for districts and states to assist in school improvement. Unfortunately, few low- 
performing schools have received intensive assistance because of limited district and state capacity 
(Le Floch, Boyle, & Therriault, 2008; Olson, 2004; Richard, 2004; U. S. Department of Education, 
2004). Furthermore, while recent research has focused on some aspects of NCLB implementation, 
such as assessment provisions and state capacity issues (e.g., Sunderman, 2008b), scant research 
exists on external assistance to low-performing schools. 
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This article seeks to inform the debate around NCLB by discussing the design and 
implementation of external support to low-performing schools under accountability policy sanctions 
based on an analysis of data from two studies: The Chicago School Probation Study and the 
Immediate Intervention/ Underperforming Schools Program (II/USP) Evaluation in California (see 
Finnigan & O’Day, 2003, and O’Day & Bitter, 2003, respectively). The accountability policies in 
Chicago and California precede NCLB by several years and thus have been referred to as “first- 
generation” accountability policies because of their design similarities compared with the federal 
legislation (Mintrop & Trujillo, 2005). 

Both of these policies recognized the need to build capacity in identified low-performing 
schools and both provided resources to do so. In particular, the policies mandated to one degree or 
another that schools work with external agents to assist them in their improvement efforts. In some 
cases, schools benefited from the assistance of an experienced and qualified external support 
provider and attributed improvements to their work with this provider. However, the two studies in 
combination suggest a key finding: The support provided to low-performing schools in most cases 
was not sufficiently targeted, coherent, or intensive to influence instmction and student learning in a 
meaningful way. As a result, the schools’ responses to external support were quite variable and most 
schools realized at most a minimal benefit from this support. These findings have important 
implications for the design of school accountability policies to ensure that low-performing schools 
receive adequate and high quality assistance that will enable them to improve. Other jurisdictions 
can learn from the experiences in Chicago and California to better design and implement policies to 
build capacity in low-performing schools under NCLB or subsequent federal legislation. 

The following sections provide an overview of the conceptual framework and literature used 
in our analysis, a discussion of the policy contexts in California and Chicago, and a description of 
the methods and data sources for the two studies. We then discuss the major results through an 
analysis of the policy strength of the two external assistance policies and the monitoring of the 
quality of support by states and districts. 


Literature 

Despite the millions of dollars spent on external change agents in school reform, research on 
their effectiveness remains sparse and inconclusive. The conceptual framework for this article is 
informed by an analysis of the underlying assumptions of school accountability policies as well as the 
literature on instructional capacity and external assistance. We begin with a discussion of one of the 
core assumptions of these policies — that support must come from the outside — and describe the 
research to date on this type of support. Next, we discuss an additional assumption, that external 
support will lead to school improvement. In that section we provide conceptualization of 
instructional capacity to frame our discussion of the design and implementation of the external 
assistance provided to low-performing schools in these contexts. Finally, we describe the analytical 
lens we use to understand the policy strength of the external support component of these policies. 

External Support 

The inclusion of external assistance in the design of these policies suggests a key 
assumption: that low-performing schools need external assistance because they lack the capacity to 
improve on their own. External organizations can potentially bring new information, perspectives, 
and resources into school communities (Datnow & Honig, 2008; Huberman, 1995). These external 
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groups may provide different types of assistance, including brokering or connecting communities to 
new ideas or practices, modeling new practices, facilitating dialogue and shared learning experiences 
around teaching and learning, and developing conceptual or practical tools (Honig & Ikemoto, 
2008). However, scant research currently exists regarding the operation or impact of these types of 
providers in schools and districts (Datnow & Honig, 2008; Supovitz, 2008). 

Investigations like the Rand Change Agent Study (Berman & McLaughlin, 1978) found little 
impact of external change agents. 1 The limited impact or effectiveness of these groups may be the 
result of support providers generally spending inadequate time in schools and lacking a strong vision 
for instructional improvement (Chimerine, Haslam, & Laguarda, 1994; Fullan, 1991). Similarly, the 
research on professional development suggests that support must be of high intensity and sufficient 
duration before it will have a measurable effect on practice (Desimone et al., 2002; Caret, Porter, 
Desimone, Birman, & Yoon, 2001; Hawley & Valli, 1998; Smylie, Bilcer, Greenberg, & Harris, 1998; 
Supovitz, & Turner, 2000). In fact, several studies suggest that the more intensive involvement of 
school “coaches” can play an important role in developing school capacity, even for schools 
identified as low performing (David, Kannapel, & McDiarmid, 2000; O’Day, Goertz, & Floden, 
1995). The limited research, to date, suggests that not only the nature of assistance and target of 
support, but also the intensity and strategic approach (or lack thereof) matter. 

Instructional Capacity 

The policies in Chicago and California assumed that these external agents would plan and 
implement programs in the schools that would result in school improvement. Past research suggests 
that to improve school-level student performance, one must focus on individual teacher capacity, 
including teachers’ knowledge of content, pedagogical content knowledge, and general pedagogical 
knowledge (Shulman, 1986). An implication of this research is that external support providers 
seeking to improve student learning should begin by focusing on the knowledge of teachers. 
However, Cohen and Ball (1999) argue that considering teacher capacity in isolation is insufficient. 
Articulating a broader view of the instructional unit, these researchers argue that instruction (and 
thus instmctional improvement) is “a function of the interaction among [teachers, students, and 
educational materials] . . . not the sole province of any single one, such as teachers’ knowledge or 
skill, or curriculum” (pp. 2—3; emphasis in original). In essence, Cohen and Ball’s examination of 
instructional improvement suggests that external support providers must move beyond developing 
teacher knowledge alone and instead leverage ways of strengthening all aspects of the instmctional 
unit. 

In addition, because the instructional unit is nested within the school organization and 
influenced by it, interaction with and management of the environment are also integral to 
instructional effectiveness. Prior research in this area has focused on the importance of 
organizational capacity, often describing particular characteristics or components of the 
environment that are necessary to bring about school improvement (Levine & Lezotte, 1990; 
Newmann & Wehlage, 1995; O’Day, Goertz, & Floden, 1995; Purkey & Smith, 1983). For external 
support providers to initiate school improvement, they must take into account each of these levels 
(the instructional unit and the school environment). At the level of the instructional unit, teachers, 
materials, and students should be the primary foci, while important targets at the organizational level 


1 Ten years later, McLaughlin (1987) found a greater impact as a result of more flexible agent-school 
relationships. 
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include coordination, professional norms, learning opportunities, resources, and the monitoring of 
student learning. 

Policy Strength 

Given the two policy assumptions discussed above (that external assistance is necessary and 
that the work of these external agents will bring about school improvement through capacity 
building), our analysis in this article examines the design and implementation of the external 
assistance provided through the school accountability policies in Chicago and California. Using a 
framework developed by Porter and his colleagues to examine curriculum policy (see Porter, Floden, 
Freeman, Schmidt, & Schwille, 1988), we consider the policy strength of the Chicago and California 
external assistance by identifying policy characteristics or aspects of implementation that are likely to 
lead to improved performance. Desimone (2002) argues that this “simple, but powerful framework” 
is useful in examining policy implementation and school reform (p. 438). 

Building on Porter et al.’s (1988) work, we examine the strength of the external assistance 
component of these accountability policies along four dimensions: specificity, consistency, authority, 
and power.“ Specificity refers to how narrowly defined a policy is. Porter et al. note that in 
curriculum policy, for example, broad standards are less specific than a more elaborated scope and 
sequence, which are less specific than a mandated curriculum and instmctional materials. 

Consistency refers to the coherence among different policies and policy elements. In this context, 
one must consider both external coherence, or coherence of the support policy with other state or 
district policies, and internal coherence, or coherence of the reforms or approaches of the external 
support provider and other providers within the school context. Policies acquire their power from 
the resources committed to their enactment as well as from the constellation of rewards and 
sanctions accompanying them and the administrative authority of those involved. In our adaptation 
of Porter et al.’s framework, we argue that the resources allocated toward the implementation of the 
policy are an integral part of the policy’s power. Policy authority may derive from varying sources 
such as professional expertise, the perception thereof, and hierarchical positions of the key 
individuals. 

It is important to note that these policy attributes are not necessarily additive or even 
independent. In reality, consistency among policies may have a multiplicative rather than a merely 
additive effect on the strength of each. In addition, policy strength may derive more from achieving 
a proper balance of the components rather than high levels of each. 

Policy Contexts: Chicago and California 

In 1996, the Chicago Public Schools (CPS) made history by placing 71 elementary and 38 
other low performing schools on probation, a judgment based on aggregate reading scores on the 
Iowa Test of Basic Skills (ITBS). Probation schools faced consequences including a decrease in 
autonomy and the threat of more severe sanctions, such as “reconstitution” or the process of 
removing all of the staff. The schools also received support from an External Partner, a Probation 
Manager, a Regional Education Officer, a Business Manager, and a District Facilitator. The district 


2 Porter et al. use the term prescriptiveness, but we believe specificity more appropriately describes 
this dimension of policy strength, particularly given recent usage of prescriptiveness in the field of education 
to refer to pre-packaged curriculum or programs. 
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fully funded the first year of external support, but in the second year the school assumed one-half 
the cost, and in the third and following years, the school was responsible for the entire amount. 

After a school was removed from probation, it was required to continue its relationship with a 
partner for one full year so that the supports were not withdrawn immediately. The External 
Partners were approved by the district and primarily university-based; that is, partners affiliated with 
local universities worked with the majority of the elementary schools. The district also assigned to 
each probation school an administrator that it judged had a proven track record of leading a school 
(though no clear criteria or definition were evident for these “successful” administrators) to serve as 
a Probation Manager and work directly with the principal. 

A few years after Chicago’s school probation policy was instituted, California state 
lawmakers established the Public Schools Accountability Act of 1999 (PSAA) with a new state 
measure called the Academic Performance Index (API) — a composite scale used to measure school 
performance. In that law was also a policy to support improvement in the state’s low-performing 
schools called the Immediate Intervention/Underperforming Schools Program (II/USP). This 
article focuses on II/USP, which provided funds to support schools’ efforts to improve but 
subjected recipient schools to future sanctions should that improvement not be realized after three 
or four years of such support. Participation was voluntary, and eligibility for II/USP involved a 
school’s ranking in the bottom half of scores statewide and failing to achieve annual growth 
expectations on the API. ’ The funding included one year for planning and two to three years for 
implementation. Implementation was funded for each participating school using either state general 
funds or federal Comprehensive School Reform Demonstration (CSRD) program funds. The law 
required that schools hire a state-approved External Evaluator during the first (planning) year to 
assist them in developing their school improvement Action Plans. Four hundred thirty schools 
participated in II/USP in each of three cohorts (1999-2000, 2000-01, and 2001-02) in this study, 
for a total of 1290 schools. 3 4 

The Chicago and California policies had similar design components. Both targeted low- 
performing schools and located the problem of low performance within the schools. In addition, the 
Chicago and California policies involved the provision of external support. In both cases, schools 
selected and contracted with independent providers with only moderate constraints by the district 
and state, respectively. Directions to external support providers were minimal, and multiple 
approaches were acceptable on the assumption that this variation would better meet the diverse 
needs of schools and would lead to greater buy-in from school staff. 5 Finally, in both places the 
policy targeted districts and schools serving primarily poor and minority populations, as documented 
by Diamond and Spillane (2004) and Mintrop (2004). 


3 In the first year of the program, the statewide performance index (API) was not yet operational. Therefore, 
schools were eligible by scoring in the bottom half of the state’s schools for two consecutive years (1998 and 1999) on 
the statewide standardized exam (SAT-9). In addition, the state included all of the schools receiving funds through the 
Comprehensive School Reform Demonstration (CSRD) program. 

4 A portion of these schools have since entered a subsequent program, the School Assistance and Intervention 
Team (SAIT) program, because they made insufficient growth during II/USP participation. In this program, schools 
work with an external, state-approved team for approximately three years to implement nine “Essential Program 
Components” (EPCs), a set of target areas for school improvement including, among others, the implementation of 
State Board-adopted curricula and the implementation of an assessment system to monitor student progress. In addition, 
California later implemented the High Priority Schools Grant Program (HPSGP), a similar program that focuses a higher 
level of funding to a smaller subset of the lowest performing schools. Approximately 290 II/USP schools later entered 
the HPSGP, thus receiving twice the dollar amount per pupil starting in the 2002—03 school year. 

5 The two policies have different titles for those groups providing external support to schools. From this point 
forward, they will be referred to as “external support providers” generally and we use the language from each policy 
when discussing specifics in each context. 
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Despite these similarities, the policies differed in a number of important ways. While 
Chicago’s probation policy was a district - level policy, the II/USP was statewide and involved urban, 
mral, and suburban schools. In addition, the criteria used to determine eligibility differed, with 
Chicago (during the course of this study) judging aggregate performance based on a cut-off score 
and California incorporating school-level performance growth. 6 Another distinction is the timing of 
assistance: California’s mandate regarding external assistance focused primarily on the planning stage 
of school improvement, and Chicago’s policy focused on both planning and implementation. 

Finally, California’s policy specified outcomes in multiple subject areas (including science, social 
studies, literacy and mathematics), while Chicago concentrated on reading scores. In spite of the 
differences, these two contexts provide important insights into the design and implementation of 
support within school accountability policies. 

Methods 

This section provides a brief description of the methods used in each study. The Chicago 
School Probation Study was conducted from 1999 through 2001 and focused on elementary schools 
on probationary status. To understand the nature, intensity, and quality of support provided to these 
schools under accountability policy sanctions, the study examined the two central components of 
the support system, the External Partners and Probation Managers. This article is based primarily on 
interviews and observations of 11 (of the 18) External Partners during this time period and 
interviews with a random sample of 16 (of the 53) Probation Managers. Other data sources include 
interviews with district administrators and facilitators, review of district documents, observations of 
support provider meetings and workshops, and interviews with a random sample of 15 principals at 
probation or post-probation elementary schools. Finally, the article used interviews and observations 
from case studies of ten schools that were placed on probation. (Half moved off probationary status 
and half remained on probation throughout the study.). 

California’s II/USP Evaluation Study occurred from 2001 through 2003. This article 
primarily draws on interviews with 21 school principals and more than 100 teachers at 21 schools in 
the study. In addition, the article uses interview data from the External Evaluators for 18 of the case 
study schools and district personnel in each of six districts associated with the case study schools. 
The information is supplemented by findings from a survey of teachers and principals in a stratified 
random sample of II/USP and matched comparison schools and survey data from 104 External 
Evaluators and 162 district personnel from districts with at least one II/USP school. In addition, a 
short follow-up study was completed in 2005 to examine longer term impacts and implementation 
of II/USP. 

The analysis for this article involved an examination of the nature and target of external 
assistance across these contexts, as well as the different aspects of the design and implementation of 
these policies relating to policy strength. The authors conducted an iterative analytical process of 
identifying and refining the emerging themes. 


6 In Chicago at the time of the study, the district identified schools based on their aggregate reading scores on 
the ITBS. In the first year schools were placed on probation if less than 15% of their students in grades three through 
eight were reading at or above national norms (the cut-off of 15% has been raised during subsequent years of 
implementation). More recently, the district used the statewide test and included a component that focuses on growth in 
achievement. In California, eligibility was determined by the ranking of the school in comparison to other schools in the 
state, as well as by the growth (or lack thereof) in student achievement across years. In California, each school was 
assigned a yearly growth expectation for the API equivalent to 5% of the difference between their current API score and 
an overall state target. 
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Results 

In this section we discuss our findings relating to the strength of the support component of 
these school accountability policies using the concept of policy strength discussed previously. Within 
our discussion, we describe the nature and targets of external assistance in terms of our 
conceptualization of instructional capacity. Finally, we link back to an overall policy assumption 
regarding the role of the district and state in providing support to low-performing schools — the 
belief that these agencies have the capacity to provide adequate and high quality support to 
individual schools. 

Strength of External Support 

Specificity. Specificity in this context refers to whether the policy provided explicit or 
detailed requirements regarding the assistance. Our findings suggest that these policies were highly 
specified about certain procedures as well as the expected outcomes, but neither policy was 
sufficiently specified about the nature and target of assistance to ensure that the external support 
providers developed coherent strategies to bring about school improvement. Both policies were 
highly specified in certain ways. For example, California’s policy prescribed activities during the 
planning year with respect to the processes the External Evaluator must use to conduct a needs 
assessment and develop an Action Plan for each school. The External Evaluator was required to 
analyze school data and gather information from parents and teachers. Both the California and 
Chicago policies were also specified in their emphasis on test scores — schools were expected to meet 
specific achievement goals in order to exit probation (in Chicago) or to avoid additional sanctions (in 
California). This specification oriented the external assistance providers’ work towards improving 
test scores. 

In both contexts, the policies for external support were vague by design in terms of the 
nature and targets of assistance. In theory, this lack of prescriptiveness allowed for individual 
matches between support providers and schools, and for improvement strategies tailored to the 
needs of the school. Indeed, a positive aspect of these policies is the recognition that schools differ, 
as do the nature of their problems. A school could choose a provider that had a similar reform 
approach (for example, a direct instruction approach versus constructivist approach), or that had 
expertise specific to the challenges at their school (for example, a high proportion of English 
Language Learners or a weak reading program). By allowing flexibility in the target and strategies for 
improvement, individualized improvement plans could be customized to the specific school needs. 

While positive in its intent, such vagueness in the nature and targets of assistance may have 
limited the policies’ overall effectiveness. For example, by relying on market forces to produce 
greater fit between schools and providers, the policies assumed that schools would select providers 
who would appropriately serve their individual needs. In reality, many schools did not have the 
capacity to know what services they needed or to determine who would provide them appropriately. 
In addition, schools typically did not know which providers had a record of success with schools like 
theirs. Thus, many schools chose a provider based on availability or location or a person whom they 
already knew, with whom they had a pre-existing relationship, or about whom they had heard from 
others. As a result, rather than creating better matches between providers and schools as intended, 
the lack of specificity around the nature of support often left schools with providers who were not 
well-suited to assist with their specific needs. 
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In addition, data from both studies indicate that the lack of specificity regarding the targets 
of support often resulted in a scattered rather than strategic orientation to improvement. The 
following discussion provides general details of what assistance looked like in these contexts; 
however, support providers varied in their particular designs. In both policy contexts, teachers were a 
primary focus. In Chicago, the external support providers worked with teachers individually or in 
group settings to increase their knowledge and skills or hone their classroom management and 
instructional practices. In California, the External Evaluators did not directly provide professional 
development or assistance in the first (planning) year, but the Evaluators targeted teachers’ 
knowledge and skills in their needs assessments and Action Plans. Nearly all of the external support 
providers also attended to instructional materials in some manner, including state or district- 
adopted curricular packages, lesson plan guides, “how-to” packets, and assessments. In addition, 
some of the external support providers focused their assistance at the organizational level. For 
example, they could have worked with a principal or a schoolwide team to strengthen leadership and 
to develop a shared vision and common goals; targeted specific organizational stmctures to facilitate 
teachers’ sharing ideas and practices; or brokered resources. Finally, in both California and Chicago 
external support providers worked with teachers to increase the use of data to monitor student 
achievement and to make instructional and organizational decisions in their classrooms. 

Our conceptualization of instructional capacity suggests that these individual and 
organizational level targets are appropriate; however, our data indicate that the assistance provided 
to schools was often scattered in its focus, not strategic. In other words, rather than targeting 
systematically and strategically the interaction of teachers and students working around particular 
content and the organizational support for such work, the providers typically focused on loosely 
connected elements of instruction or school environment. In addition, the target of assistance varied 
from provider to provider with no real guidance from the policy authority. This variation in target 
could be beneficial if the assistance was consistently aligned to the real needs of the schools or 
directed clearly toward the improvement of instruction and student learning; however, our data 
suggest otherwise. For example, in California, many schools rewrote, revised, or ignored the Action 
Plans developed in the planning year by the External Evaluator if their staff did not think the plan 
met their (often changing) needs. 

The variation in targets and design of support was compounded in California by a lack of 
specificity for the external support provider’s role in the implementation years of the policy. While 
the policy provided specific guidance concerning the work of the External Evaluator in the planning 
year (which, depending on the timing of funds, ranged from just a few months to nearly a whole 
school year). External Evaluators were neither required to assist the school nor monitor 
implementation beyond that first year. In fact, schools were initially discouraged from using the 
same External Evaluator during the planning and implementation years for fear that the External 
Evaluator’s work in the initial phase would be biased toward identifying strategies that matched their 
own programs rather than the school’s needs. Thus, again, the policy was designed to provide 
flexibility for schools in the use of their implementation funds. However, in the end, schools were 
left with little direction on how to implement their Action Plans and on whom to call for assistance 
in doing so. In fact, 77% of surveyed II/USP principals said they had received limited or no follow- 
up from their External Evaluators during implementation. This disconnect between the specificity in 
the planning year and years to follow likely hindered the implementation of plans significantly. The 
Evaluators themselves acknowledged this shortcoming: 63% of surveyed External Evaluators 
responded that confining the External Evaluator’s role to the first (planning) year of II/USP was not 
useful. In fact, the II/USP Evaluation found no clear connection between the quality of the 
External Evaluator’s work during the planning year and subsequent implementation of plans and 
growth in student outcomes. 
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Our analysis of the specificity of the California and Chicago policies raises the question of 
which aspects of a policy should be highly specified and which should remain vague or unspecified. 
Both policies while narrowly defining the expected outcomes did not specify the focus of assistance. 
First, there was no explicit guidance to link planning to practice and changes in instmction. Second, 
little direction was provided in either policy context around the ways in which improvement goals 
set by schools and external assistance providers should directly relate to instruction. Thus, plans and 
strategies often did not tie into a coherent set of instructional goals (see discussion of consistency 
and coherence below). In fact, the external support providers’ strategies rarely involved coherent 
planning or targeted implementation and, as such, the external strategies frequently deposited an 
additional layer of reforms in these schools. In Chicago, most external support providers offered in- 
service training to teachers that emphasized generic pedagogical and management strategies, rather 
than strategies linked to particular content areas and to goals or standards in those areas. 
Furthermore, most of these providers did not emphasize (or assist teachers and administrators in 
developing) instructional coherence as a key goal (see more details below). As a result, the support to 
low-performing schools in Chicago through this policy was not likely to result in deep instructional 
improvement. Similarly, as mentioned above, a lack of specificity in the external support provider’s 
role regarding implementation of Action Plans in California resulted in little connection between the 
external support providers’ work in the planning year and actual changes in instruction and student 
outcomes. 7 Even though External Evaluators had the opportunity to provide insights about ways to 
improve instmction across subject areas during the planning year, their insights were only 
meaningful if the school followed through on actions to address them. The ability to follow through 
was hindered by the division between planning and implementation. 

In some cases, specificity was infused through another entity. For example, some districts in 
California required their schools to use the same highly specified curricula or instructional materials. 
These policies provided more specific guidance in terms of instmctional goals and improvement 
strategies, and indeed, we found that those schools that were most successful in implementing a 
coherent instructional program had strategies that were at least specified enough to get all teachers 
on the same page. Unfortunately, the districts’ more prescriptive guidance sometimes conflicted with 
reforms put in place at the schools through II/USP. We discuss the effect of this specificity on the 
policy’s consistency in more detail in the following section. 

Consistency. Coherence at the school level is important to the improvement of low- 
performing schools as it ensures clear goals and fosters the alignment of school strategies and 
resources to these goals (Newmann, Smith, Allensworth, & Bryk, 2001). We found that only four (of 
10) case study schools in the Chicago study and eight (of 21) case study schools in California had 
coherent instructional programs. In California we found a strong relationship in our case study 
schools between the presence of a coherent instructional program and improvements in student 
achievement outcomes — that is, those schools with more coherent programs also demonstrated 
greater and more consistent gains in student test scores. In a few cases, the coherence appeared to 
develop, as intended, through the needs assessment and planning process of II/USP, with help from 
the External Evaluator. Flowever, in other cases this coherence had clearly been forged in spite of 
rather than because of the work of the external provider. Both studies found that the external 
support providers’ programs were often considered add-ons to the schools’ core work and were 
rarely integrated into the schools’ overall goals and reform efforts. This add-on nature of support 
resulted in a lack of internal consistency around the reforms or approaches implemented at the 


7 In contrast, in the SAIT program, the SAIT providers were required to organize their work around a set of 
nine “essential components” for school improvement, set out by the state. These components increased the specificity 
of the SAIT’s work in both the planning and implementation phases. 
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schools. In other words, the support often did not become part of an integrated, institutionalized 
improvement strategy. Our findings suggest that this dimension of the policies was weak for the 
following reasons: the lack of fit between external support provider and schools; the fragmentation 
between planning and implementation; the lack of coordination among multiple agents in these 
schools, and other conflicting policies implemented in these contexts. 

In fact, inconsistency or lack of coherence in support providers’ work was a substantial 
design and implementation issue in both California and Chicago and limited the coherence of the 
resultant improvement strategies and instructional reforms. Though not explicitly written in the 
legislation, California’s policy attempted to foster instructional coherence within the schools through 
the planning year; the external support provider was expected to facilitate the development of a 
coordinated plan for improvement during this year. In Chicago, coherence was not explicitly 
addressed but was promoted indirectly through the school improvement plans that probation 
schools were required to submit and Probation Managers were required to monitor. 

In several ways, the lack of integration discussed above resulted from the policies’ limited 
specificity around instruction, as described in the previous section. For example, one school in 
Chicago with a constructivist approach to learning hired an external support provider associated 
with the Direct Instruction program because of the strong reputation of the program director. Since 
the school intended to continue with its constructivist learning philosophy, the external support 
provider’s program was in conflict with the school’s core program and agreed-upon philosophy of 
teaching and learning. Direct Instmction was not adopted programmatically, although the provider 
continued as the school’s official External Partner. The conflict was exacerbated by the fact that the 
external support provider appeared to do little to understand and adapt its assistance to meet the 
individual needs of this school. The school’s performance levels improved rather quickly, but the 
improvements seemed to have nothing to do with the policy’s external assistance, which school 
personnel viewed as peripheral to the changes that led to improved performance. In California, the 
policy provided little guidance with regard to the instructional content focus of the Action Plan. 
Thus, in several cases, external support providers developed plans for improvement that included a 
range of disconnected strategies. For example, in one rural middle school, the Action Plan included 
35 strategies for teaching, learning, and staff development, while seven strategies focused on 
governance. This laundry list of actions to address a long list of identified barriers was a challenge 
for staff to implement and was therefore essentially ignored. 

In California, consistency of the policy was also weakened by the disconnect between the 
planning and the implementation years, as described earlier. The lack of a specified support 
component in the implementation years weakened the coherence of both the support and the 
resultant improvement strategies. Since the planning process was divorced from implementation, 
external support providers generally lacked a commitment to the school and to the implementation 
of the plan. This aspect of California’s policy design limited schools from following through with 
their school improvement plans in coherent ways. 

Beyond the official external support provider associated with these policies, multiple other 
agents often assisted these schools. Communication among these various agents was minimal, 
leaving little opportunity for these multiple groups to coordinate their efforts and assist the school in 
a coherent way. When various groups involved in the same effort have contradictory messages, each 
group’s effectiveness is severely limited (Hatch, 1998). In this scenario, the existence of multiple 
groups heightened the probability that school staff would be forced to contend with conflicting 
messages about school improvement and, as a result, opportunities for real improvement would be 
duplicated, contradicted, or lost. The school improvement efforts of these external support 
providers frequently became peripheral or irrelevant, especially when they were the newest partner 
to work with these schools. 
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On a similar note, to foster coherence at the school level, a policy must be explicitly 
designed to engender that coherence. To do so, the policy itself needs to be consistent with other 
existing policies to avoid pulling schools in multiple directions. In both California and Chicago such 
policy coherence was limited. Specifically, district policies sometimes hindered the ability of schools 
to utilize fully the external assistance they received, particularly when those district policies were 
specified in ways that contradicted the assistance provided through the II/USP or probation 
programs. For example, in California, two case study schools as part of their participation in II/USP 
(CSRD) adopted whole school reform models that emphasized the development of the whole child 
and incorporated art as a means of doing so. However, these models contradicted the subsequently 
adopted scripted reading and mathematics programs for their districts. Unable to ignore the district 
requirements, the schools pulled away from the reform model, though it remained in the school as a 
source of incoherence and distraction rather than as a comprehensive reform design. 

Similarly, in Chicago, the district began recommending that low-performing schools use 
structured reading lessons provided by the central office. In part, this district policy was a response 
to the recognition that schools needed additional guidance with respect to instruction and was 
intended to provide schools with instructional tools. Unfortunately, these stmctured lessons were 
sometimes in conflict with the External Partners’ approach to reading, resulting in confusion 
regarding how to align these disparate strategies. The district also instituted a policy for schools that 
received additional federal class-size reduction funds through the Targeted Assistance Program, 
requiring that students be instructed using only grade-level reading materials during the literacy 
block. This policy was in direct conflict with that of some external partners’ programs (e.g.. Success 
for All) that focused on flexible grouping across classrooms and use of literacy materials at the 
student’s instructional level rather than only on a uniform grade level. Even where the external 
partners’ programs appeared to be improving student literacy performance, schools were forced to 
abandon them in favor of the uniform, grade-level approach. These types of contradictions arise 
when the multiple policies to which schools are subject are not aligned or when key stakeholders, 
such as the district in California’s case, are not involved in designing the policy. When examining 
support between the district and external organizations discussed by Supovitz (2008), both Chicago 
and California exhibited fragmentation rather than the melding. 

While the previous discussion focuses on coherence and consistency at the school level, 
inconsistencies across the state (California) and district (Chicago) also existed. For example, in 
California, the external support providers’ skills and work were not consistent across II/USP 
schools. As discussed previously, the targets of the providers varied considerably, with some 
focusing on instructional materials, and others focusing on professional development. This variation 
was intentional, to foster reforms more tailored to individual schools’ needs. However, in both 
contexts, there was little to no networking or training to ensure that external support providers were 
consistent, resulting in a wide range of experiences among schools in the planning year, and a broad 
set of reforms implemented sporadically across the state. 8 We also found a range of expertise and 
commitment of the external support providers to these low-performing schools. In both contexts, 
an approved list of external support providers was developed but the selection process did not 
ensure a minimum level of quality, and little to no monitoring of the providers’ work occurred as 
they assisted schools. As mentioned previously, the support providers were mostly selected based on 
prior relationships rather than a fit with the school’s needs or evidence of the quality of support to 
be provided. 


8 In California, this problem has been mitigated somewhat in the SAIT process, since all external providers 
attend a common training and have a common template to use to plan strategies for improvement. 
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Power. The Chicago and California policies manifested their power through large financial 
commitments and threatened sanctions to participating schools. Despite these large financial 
commitments in terms of aggregate dollars for the program, our findings suggest that the policies’ 
power was limited by the low intensity and limited resources associated with the external support 
providers’ work; limited implementation of the consequences associated with the policies; and a low 
level of administrative authority. These limitations in power lowered the salience of the policy 
among school staff and limited the ability of the policies to influence classroom instruction. 

Significant funding was allocated toward both policies. In Chicago, the district provided the 
funding in full for the External Partners in the first year of probation and paid for half of the 
support during the second year. By the third year the district no longer paid for the assistance; the 
schools were required to incur these costs. Many schools began using federal Comprehensive School 
Reform Demonstration Program (CSRD) grants, which they received through the district, to pay for 
their portion of the support after the first year on probation. The district also paid the small stipend 
to Probation Managers every year. In California, schools were allocated a planning grant of $50,000 
in the first year to be used for planning activities and hiring the External Evaluator. During each of 
the subsequent implementation years, schools received $200 per pupil. Funds were provided by the 
state and included federal CSRD funds, and schools were expected to allocate $200 per pupil to 
match the II/USP grants (typically using Title I funds). 

The research indicates that external support must be of high intensity if it is to improve 
educator and organizational capacity (Hawley & Valli, 1998; Smylie, et al., 1998; Supovitz & Turner, 
2000). In these low capacity schools, the external support providers were often stretched too thin to 
provide the necessary support to teachers or principals to generate the level of improvement needed. 
All external support providers in Chicago reported observing teachers in their classrooms, but this 
assistance varied in quality and was generally of low intensity — approximately one or two days a 
week for the whole school for External Partners and two to three days a month for Probation 
Managers’ work with the principals. The number of teachers receiving direct assistance from 
external support providers was difficult to determine through our interviews and observations and 
seemed to change on a regular basis. Many External Partners spent approximately one day a week at 
a school focusing on multiple tasks during that time, including observing teachers in classrooms, 
providing feedback to teachers, and distributing materials. In some cases, more intensive support 
was provided to a small number of staff. An exception to this low level of intensity was evident in 
one of the Chicago schools. In this school the external support provider’s program involved the 
addition of an on-site staff person to provide continuous support and professional development to 
teachers, in conjunction with a coherent literacy program in which the on-site coach received 
intensive training. This staff literacy coach was in frequent contact with consultants from the 
external support provider. Over time, the school developed a system in which teachers at every 
grade level met several times a week to plan, and during one of these planning sessions they were 
required to meet with the on-site support person to discuss curriculum and instructional issues or 
challenges. This type of institutionalized support and school-level capacity building resulting from 
the partnership with the external support provider was rare. Instead, many support providers in 
Chicago continued to reflect the conventional model of fragmented, one-shot sessions and trainings 
during half-day and full-day workshops rather than intensive and ongoing support. 

In California, some of the External Evaluators spent only a few days in total at the schools. 
In addition, some providers were spread too thin across multiple schools to adequately identify 
needs among teachers and the effective and comprehensive strategies necessary to address them. At 
one school, staff said that the External Evaluator “took the money and ran.” According to the 
teachers, the Evaluator did what was required by law but did not actually incorporate any feedback 
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from teachers, using primarily documents that were already written to prepare the Action Plan. 

While this was an extreme case, these teachers’ complaints of seeing too little of the External 
Evaluator were by no means unique among participating schools. At another school, for example, 
three out of four teachers interviewed would have liked to have seen more of the External 
Evaluating group. “They leave and don’t come back,” one teacher said. The teachers also reported 
that the External Evaluators were not in the classroom enough when they were at the school. 

Another aspect of policy power lies in the incentive mechanisms incorporated into 
implementation. These incentives may be either positive (rewards) or negative (sanctions). While 
Chicago and California included incentive components in their accountability policies, the data 
indicated low salience or motivational power associated with these incentives in both contexts. In 
Chicago, the threat of sanctions was not the only policy consequence. Indeed, identification as a 
probation school (note the punitive name: probation) was itself considered a sanction by most 
school personnel. The more severe sanction outlined in the policy, however, was the threat of 
reconstitution or re-engineering of schools that did not move out of the probation status, both of 
which involved removal of school staff. During our study, elementary schools that had been on 
probation four or five years were unclear about the consequences they faced if they did not reach 
the performance targets. In 1996 seven CPS high schools were reconstituted, but no elementary 
schools had been reconstituted through this policy at the time of the study. Uncertainty, a general 
sense that there was little the district could actually do to them, and the lack of positive rewards 
severely limited the salience of the policy consequences. 9 This finding is similar to that in Mintrop’s 
(2004) study of the response to probation in Kentucky and Maryland, in which educators 
disregarded the threat of sanctions contending that they were not credible responses. This is not to 
say that there was no motivational impact to probation in Chicago. Certainly, probation brought 
increased oversight from the district and the stigma of the probation label. While these were not the 
severe sanctions that were to result from the policy, these negative incentives proved rather strong in 
practice for many school staff in Chicago who believed that their professionalism was being called 
into question by their peers at other schools or in the broader community (Finnigan & Gross, 2007). 

Similar reactions to sanctions were evident in California. While school staff generally 
recognized that the policy opened their school to sanctions, few expected the state to implement the 
most severe sanctions (e.g., school takeover or closure, or the firing of staff or administrators). Less 
than half of the surveyed teachers participating in II/USP expected that these severe sanctions 
would occur, while a higher percentage of teachers expected less severe sanctions such as a public 
hearing or the assignment of an assistance team. Like Chicago’s probation policy, however, the 
II/USP did initially hold a negative stigma due to the label of underperforming and attribution of 
underperformance to the school and staff. 

The power of these sanctions was not universal in Chicago and California and, in fact, the 
effect of these dissipated over time as schools began to see that nothing concrete happened as a 
result of extended identification. For example, in California, over half of the case study schools later 
viewed participation more positively as an opportunity for change. At the time of the Chicago study, 
a number of schools had been on probation for five or more years, and the district was only 
beginning to discuss potential sanctions if they did not show improvement. The consequences for 
these types of schools that lingered on accountability sanctions were not addressed in either context 
and have not yet been addressed under NCLB as schools remain under the final restructuring stage 
(Center on Education Policy, 2008). 


9 We are not suggesting that the sanctions were appropriate, but are noting that the power of the policy 
decreases without rewards or sanctions for those involved. 
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At the support level, the policies were rather weak in terms of the authority of support 
providers to ensure implementation of improvement strategies. In both contexts, the external 
support providers had no line authority; that is, they had no supervisory power over the schools. 
They could not force the schools to implement their programs, hire or fire teachers or 
administrators, or require that certain teachers receive their assistance. Nor did the relevant school 
bureaucracy use its administrative authority to press principals toward more effective 
implementation of the providers’ assistance. The only providers that had any degree of 
administrative authority were the Probation Managers in Chicago, who monitored the schools’ 
implementation of their improvement plans. The monitoring role was only vaguely defined, 

however, and few Probation Managers believed that their responsibilities extended beyond principal 

10 

mentoring. 

Authority. For support policies associated with school accountability programs to produce 
their desired effects on schools, school personnel must perceive and accept the external support 
providers as knowledgeable experts — that is, as specialists who could help the schools diagnose 
problems and plan for school improvement. Since the external support providers did not have 
administrative power, they had to rely on professional authority. Unfortunately, the variable quality 
of assistance threatened the ability of these groups to win the professional confidence of audiences 
at the school sites. Our findings suggest that the vendor relationship between the external support 
providers and schools, inadequate selection processes, concerns about the quality or appropriateness 
of the providers’ support, and the resistance of school staff and subsequent compliance resulted in 
the low professional authority of the external support providers in most of these schools. 

The external support providers with the highest level of bureaucratic authority and varied 
levels of professional authority were the Probation Managers in Chicago. Probation Managers 
worked directly with the principals, mentoring them in a variety of areas. In some schools, the 
Probation Managers’ high levels of professional authority were linked to their own experiences in 
the Chicago Public Schools, although in some cases the probation school principals complained that 
these administrators came from different types of schools and, therefore, had limited understanding 
of the situation at hand. While there were instances of the external support providers commanding 
high levels of professional authority among teachers, our findings suggest that this was less evident 
for the reasons described in the following paragraphs. 

The support providers in both California and Chicago served in a vendor relationship, where 
schools hired them for a specific, limited service, rather than joining the schools in a partnership 
arrangement. This vendor relationship meant that their ability to carry professional authority at the 
school site was even more critical in maintaining their status (and funding) as the school’s external 
support provider. While some teachers in both contexts discussed the benefit of the policies’ 
external support components in terms of the extra tools they received, opportunities to try 
something new, training, or new perspectives, a frequent complaint in a few Chicago schools was 
teachers’ feeling that their own professional authority was diminished by the requirement to have 
someone from the outside teach them how to teach. As one teacher said, “ [They’re telling us] ‘y ou 
guys don’t know what you are doing so we have to have some outsiders come in here and tell you 
what to do.’ [These are] people that have never been in the classroom or have been in it 20 years ago 
and they’ll tell you what to do.” This scenario was observed in a California school where teachers 


10 The power of the policy in California has increased over time through the SAIT process. Monitoring by state 
officials is an important component of this process with specified expectations for monitoring meetings and reports, as 
well as requirements with regards to implementation benchmarks. This increased level of administrative authority over 
the schools strengthened the power of this policy. 
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complained that the External Evaluator focused too much on the negative and did not acknowledge 
the work and skills of teachers who had been struggling with the identified issues all along. 

Inadequate selection processes further hindered the appropriate matching of providers with 
schools and thus lowered the professional authority of the support providers. In most schools the 
staff did not attempt to thoroughly review their programs and data, identify the core problems in 
their schools, and select an appropriate partner based on these analyses. Instead, many schools made 
decisions to select particular partners based on personal connections or previous relationships. 
Similarly, decisions in California did not seem to result from a clear diagnosis of the problem and 
appropriate external support provider; however, this was primarily due to time restrictions and the 
limited pool of providers. Several schools reported that it was difficult to locate an available, 
approved provider. 

Resultant concerns regarding the quality of assistance provided and the extent to which it 
was adapted to local circumstances further diminished the professional authority of the providers. 
Low professional authority in Chicago was associated with the perception that the external support 
providers’ intervention was not clearly defined. As a result, school staff did not fully understand 
what expertise or services they were buying and even how to differentiate among support providers 
when determining which to select. Low professional authority in both California and Chicago was 
linked to concerns within the schools regarding the capacity of the external support providers to 
offer the promised assistance. In some instances external support providers were forced to hire 
additional staff using lower standards to ensure that they could work with the high number of 
schools they were assisting. Furthermore, both Chicago and California experienced a high degree of 
turnover in support providers, resulting in a lack of continuity for schools. Finally, in both contexts, 
school staff complained about the quality of the assistance. 11 The fact that the district and state did 
not have rigorous selection processes, nor did they monitor the assistance provided by these groups, 
further weakened the professional authority. 

Finally, in both California and Chicago, professional authority was called into question when 
the support providers proposed strategies, made recommendations, or promoted their particular 
programs without identifying ways in which these should be adapted to local contexts. This finding 
is in clear contrast to Honig & Ikemoto’s (2008) finding that the external provider in their study 
continuously revisited or altered their activities based on local conditions and needs. In Chicago, 
staff in several schools questioned whether the support providers understood the extreme 
circumstances they faced in these schools and communities. Similarly, in California, school staff in 
mral settings questioned whether their External Evaluators understood their unique, mral context. 
This criticism was linked to the low level of intensity of the support (discussed previously) because 
school staff believed that these individuals simply were not at their schools enough to understand 
their schools. 

Overall, in the Chicago and California cases where the professional authority of the external 
support providers was high and the benefit of what they were offering was clear, school staff were 
likely to implement their programs or recommendations. If the support providers were not viewed 
as legitimate — in other words, if the professional authority of these providers was low — school staff 
were likely to resist, adapt, or simply not implement the programs or recommendations the 
providers offered. In some of the schools, teachers and principals did not buy in to the external 

11 The SAIT process in California provides an interesting contrast to these scenarios. A combination of factors 
in the SAIT process served to increase the professional authority of the external support providers. First, the state 
implemented more rigorous selection processes for providers to ensure they had experience turning around low- 
performing schools. Also, county offices of education could serve as SAITs. Several respondents noted that the county 
was very familiar with the schools they served, and therefore could more adequately understand and address their 
challenges. Finally, the process for SAIT involved more rigorous training required by the state. 
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support providers’ programs, while in others they primarily reacted to the mandatory nature of the 
policy and associated stigma of being identified as a low-performing school. Regardless of the 
reason, their resulting resistance meant that the programs were never fully integrated into their 
classrooms and schools. For example, in one II/USP case study school in California where a 
subgroup of teachers resisted implementing a whole school reform model recommended by the 
External Evaluator, the school made participation in this model and the associated external support 
optional. As a result, he reform did not result in a universal, coherent strategy through the school. 

As one teacher in Chicago described, “[The university-based external support provider] has a 
program that they use in all the schools they work with. [The provider] was brought in with no input 
from the faculty. I don’t remember being asked about it. Maybe we were but I don’t think so. It 
worked out for me because I did things like that before but others may do things differently. They 
were forced to change.” This teacher does not explicitly discuss the resistance of teachers but 
provides insights into the lens teachers used to determine whether they accepted the external 
support provider’s approach that had to do, in part, with the actual instructional strategies (and 
whether or not they were already doing it) and, in part on the process for determining who would 
provide assistance as a result of this policy. 

In Chicago, perhaps linked to this resistance, as well as the mandatory nature of probation 
and external support, most schools focused on compliance. Principals played a critical role in 
conveying their own views of the professional authority of the external support providers and 
overall policy. For example, in one Chicago school the principal viewed neither the policy itself nor 
the support component as legitimate. As a result, his school became an example of minimal 
compliance with the External Partner requirement. Rather than valuing what the support provider 
could offer in terms of professional expertise, he had the provider focus on a very narrow aspect of 
the school’s program: the retrofitting of the school’s wiring for technology. The lack of specificity in 
the policy with regard to focusing the providers’ work on instructional support contributed to this 
principal’s adaptation of the support mechanism. As mentioned previously, another of the Chicago 
case study schools chose an external support provider with a philosophy of learning and instruction 
that contradicted the school’s philosophy, illustrating in a different way the limited value placed on 
the role of the External Partner in the school’s improvement efforts. Interestingly, at least minimal 
compliance occurred across the board in Chicago in spite of the fact that there were no clear 
consequences resulting from non-compliance. 

Limited Monitoring of External Support Quality 

A key problem in both Chicago and California was the limited capacity of the district and 
state, respectively, to directly assist the large number of low-performing schools. More recently this 
same problem has been found in response to NCLB as a result of low levels of funding and 
increased (and shifting) responsibilities of these agencies in addressing the problem of low 
performance (Sunderman & Orfield, 2008). According to the U. S. Department of Education (2004), 
districts, and not states were most often providing assistance to low-performing schools, yet most 
districts were unable to provide the resource-intensive support necessary because of their own 
limited capacity. In Chicago and California, limited capacity resulted in a reliance on external support 
providers to bring about school improvement. Therefore, it was of utmost importance that these 
providers were carefully selected and monitored to ensure that they provided high quality assistance. 
Unfortunately, in these contexts district (Chicago) and state (California) administrators had limited 
capacity to oversee external support providers, with little screening of providers or monitoring and 
evaluation of assistance occurring. While California did improve the rigor of the External Evaluator 
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selection process over time, they did not implement a process to directly monitor or evaluate 
individual support providers’ work. As one teacher said, “[The state] needs to look at [the External 
Evaluators] much more closely. In some cases there are people on the list to be consultants who are 
not at all able to carry out that function.” In other words, while the schools faced sanctions for low 
performance, there were no sanctions applied to external support providers if they offered 
inadequate or poor quality assistance. The capacity of state and district agencies to even provide this 
most basic oversight of external support providers, not to mention the provision of direct assistance 
to their low-performing schools that is assumed by NCLB, must be addressed for schools to receive 
adequate support and to ensure that these policies address the central assumption that low 
performance is linked to school capacity. 


Conclusions 

The Chicago and California policies represented important advancements with regard to 
building the capacity of educators in low-performing schools. However, these studies suggest 
fundamental problems in the design and implementation of school improvement support 
mechanisms that must be addressed if this support is to truly benefit the recipient schools. Weak 
policy strength and limited monitoring of external providers’ quality resulted in the support 
component of these policies having little influence on teaching and learning. While some low- 
performing schools benefited from the external assistance, perhaps as a result of the higher levels of 
initial capacity within the school (Gwynne & Easton, 2001), the majority continued to struggle 
despite additional assistance. The limited benefit of this support may not be surprising given the 
decades of problems and neglect that many schools faced. 

The implementation of both policies relied on market mechanisms to control the quality of 
external support providers. However, most selections of partners had little to do with the quality (or 
even the fit) of their programs. Combined with the variability in the nature of support, this fact 
meant that a large proportion of schools did not receive appropriate or high quality support targeted 
toward their needs. External support providers’ assistance should be integrated into a 
comprehensive, strategic approach to school improvement linked to individual school problems. 
Furthermore, at the heart of school improvement is instructional change; therefore, the external 
support providers should be required to demonstrate a link between their programs and 
instructional improvements. Schools must be provided with more information before selecting 
partners to allow them to make better selections (although the capacity of school staff to identify the 
root of their performance problems remains an issue). In addition, the district or state must evaluate 
the assistance of the support providers to ensure that they not only meet the needs of schools they 
serve, but also are held accountable for the implementation and outcomes of their work. In both 
California and Chicago, the external support providers were not affected in any way if the schools 
they assisted continued to perform at low levels: They were not removed from the list of approved 
agencies, nor even oftentimes the school itself. 1 ' 

While these policies target the problem of low performance inside the school, the root of 
low performance rarely is located in the school alone. Instead, many school problems are related to 
larger, systemic issues and policies at the district and state levels. Low-performing schools in both 
Chicago and California were dealing with decades of problems that were beyond the schools’ 


12 While California did remove some providers when preparing a revised list for later cohorts, 
providers were chosen based on their applications rather than on an analysis of the work and success they had 
with prior II/USP schools. 
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control. For example, these schools typically faced greater turnover in staff and on average had less 
experienced school personnel than higher performing schools (Allen, 2005). As Mintrop (2004) 
describes, these are challenging work environments. Successfully turning around low-performing 
schools requires an examination of both the problems relating to the internal functioning of the 
school organization, as well as systemic factors (policies and practices within and outside of the 
school system) contributing to low school performance. 

Finally, the strength of this policy is closely linked to the resources available for supporting 
schools. As McLaughlin (1987) argues, a combination of pressure and support is critical. Yet, our 
studies indicate that the support components were so weakened that only the pressure was having an 
effect, albeit limited, on school staff in Chicago. Without the high levels of intensity required for the 
support to have a marked effect on school performance, the support component of these policies 
was simply not adequate. This was particularly problematic for the lowest capacity schools, many of 
which experienced little change in performance despite the assistance they received. In essence, 
simply requiring assistance does little to improve educator and organizational performance. 

Whatever its original design, a policy is transformed as agents at multiple levels of the system 
interpret, adapt, and act (Elmore, 1987; Spillane, 2000; Weatherly & Lipsky, 1977). In Chicago and 
California, these policies were designed to improve schools through professionals in the field and 
were a promising way of addressing the limited capacity of the district and state by relying on outside 
experts and encouraging the investment of these experts in turning around low-performing schools. 
However, these attempts met with minimal success, in part because individuals at all levels of the 
support system lacked the capacity to ensure that the appropriate assistance was provided through 
means such as effective selection processes for external support providers and strong monitoring 
and evaluating of the assistance provided. To date, NCLB implementation has not focused on this 
critical area of school, district, and state capacity-building (Sunderman, 2008a). 

The findings from Chicago and California are important to our nation’s understanding of 
what is required of schools, district, states, and external agencies in the improvement of low- 
performing schools. While NCLB, like the Chicago and California policies, requires support from 
external groups to assist low-performing schools in school improvement, this support will likely 
have little impact on teaching and learning unless the support component is more clearly specified to 
focus on instruction; a more rigorous screening process exists at the school or district level to ensure 
a coherent fit between school needs and support provider programs; substantial resources are 
available (perhaps targeted toward those schools with the greatest needs) to allow for the necessary 
intensity of support at the school site; monitoring of the support providers occurs to lessen the 
variation in the quality of support across schools; and district and state practices that contribute to 
persistent low performance in some schools are addressed and corrected. In sum, our findings 
suggest that improving low-performing schools will only occur after a focused diagnosis of school 
needs, matching of these needs with particular external support providers, and intensive and 
sustained support by these groups. Rather than just brokering this assistance, states and districts 
must take a more active role in building school-level capacity and monitoring external support 
provider quality; otherwise, the support component of NCLB will likely have little impact on school 
performance. 
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